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Abstract Title 
Modifying image data 



(57} An image data modifying system has devices for image storage, display, processing and manual control. 
The storage device is arranged to store many modifying textures in the form of stamps having fewer pixels 
than a full image frame. The processing device constructs a new texture stamp by scaling a first stamping 
texture, expanding a second stamping texture without scaling and combining the scaled texture with the 
expanded texture. In this way, the effect of scaling artefacts is significantly reduced. 
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1 

Modifying Image Data 

The present invention rentes to image data modifying apparatus, 
comprising image storage means, image display means, processing means 
and a manually controllable input means. In addition, the present invention 
relates to a method of modifying image data in which pixel values in an image 
are modified in response to manual operation of an input device. 

The manual modification of image data within a data processing 
environment or video editing environment etc. is commonly referred to as 
"painting". While no physical paint is actually involved in this process, the 
operation performed by an artist using a mouse, stylus or similar manually 
operable device, is similar to the action of applying a brush to a canvas and 
measures are often adopted within the environment to copy the response of 
brushes and similar graphical tools. 

Painting processes are often implemented by performing a sequence 
of stamping operations in which a region of pixels lying underneath the 
notional footprint of a stamp are modified in response to values stored within 
a stamp patch, in addition to other control values which may be derived from 
stylus pressure etc. It is also known to store a plurality of stamp patches, 
each representing a different increment of sub-pixel brush movement, such 
that the effect of a natural brush stroke with a definition in excess of pixel 
resolution may be accurately simulated. 

As the definition of images increases, such as when modifying high 
definition video or when modifying images derived from cinematographic film, 
the number of pixels covered within a brush stamp will also increase 
proportionally and, in systems where a particular profile is recorded for each 
possible size, a significant number of brush profiles will need to be made 
available; although it is unlikely that all of these would actually be required 
for a particular job. 



In order to overcome this problem and to avoid the need for storing a 
very large number of brush profiles, it is known to store an example of the 
brush profile representing a particular shape and to produce the range of 
sizes by scaling the brush profile. Furthermore, it is also known to manipulate 
brush profiles in the form of textures which may then be rendered into an 
existing frame buffer image at sub-pixel resolution. Thus, in this way, it is 
possible to store a single brush profile which may be used to produce high 
definition sub-pixel stamping in addition to producing a plurality of brush 
sizes, thereby drastically reducing the number of brush profiles which need to 
be stored. Thus, in a typical environment, it becomes only necessary to store 
a new brush texture if a differently shaped brush profile is required. Thus, 
brush profiles may be provided for simulating the effects of air brushes, 
stipple brushes, chalk, paint brushes and pencils etc. and the same data may 
be scaled and positioned at sub-pixel resolution so as to provide smooth anti- 
aliased stamping. 

A problem with this approach is that as a brush stamp is scaled, many 
pixel values within the brush stamp will be given similar intensity values. As a 
brush of this type is used repeatedly, unwanted artefacts between brush 
values may become visible and possibly displayed in the form of a series of 
rings. Furthermore, the availability of processing resource for mitigating the 
effect of these rings is limited, given that the pixel processing is required in 
real time, in response to manual indications made by an operator. 

According to a first aspect of the present invention, there is provided 
an image data modifying apparatus comprising image storage means, image 
display means, processing means and a manually controllable input means, 
wherein said storage means is arranged to store a plurality of modifying 
textures in the form of stamps having fewer pixels than a full image frame; 
and said processing means is configured to construct new texture stamps by 
scaling a first stamping texture, expanding a second stamping texture without 
scaling and combining said scaled and said expanded textures. 



In a preferred embodiment, the first stamping texture represents a 
brush profile and said storage means may be configured to store a plurality of 
brush profiles. Preferably, the second stamping texture represents random 
noise and said random noise may have a relatively low amplitude. 

According to a second aspect of the present invention, there is 
provided a method of modifying image data, in which pixel values in an 
image are modified in response to manual operation of an input device, 
comprising steps of storing a first stamping texture, storing a second 
stamping texture and producing a new stamping texture by scaling said first 
stamping texture while expanding said second stamping texture without 
scaling. 

Preferably, a new texture is rendered into an image at sub-pixel 
definition. 

The invention will now be described by way of example only, with 
reference to the accompanying figures, in which: 

Figure 1 illustrates a video artist performing a painting operation upon 
a video frame, including a manually operable device and a visual display unit; 

Figure 2 details a main screen shown on the visual display unit, 
including buttons for selecting modes of operation, including a set-up mode; 

Figure 3 details an image displayed on the visual display unit in 
response to selecting the set-up mode identified in Figure 2; 

Figure 4 illustrates painting stamps being applied to the displayed 
canvas shown in Figure 2; 

Figure 5 illustrates a pixel texture map making up a relatively small 
stamp of the type shown in Figure 4\ 

Figure 6 shows how the relatively large stamp shown in Figure 4 is 
generated from the smaller texture map illustrated in Figure 5; 

Figure 7 illustrates a noise texture map formed by tiling smaller texture 
maps; and 



Figure & illustrates how the scaled bush profile and the tiled random 
texture are combined in a painting operation performed by the apparatus 

illustrated in Figure 1. 

A facility for performing "painting" operations upon digital image data is 
illustrated in Figure 1. A processing system 101 is configured to store digital 
representations of image frames such that individual frame data may be 
displayed to an operator by means of a visual display unit 102. In response to 
this displayed image, an operator provides input data to the processing 
system 101 via a manually operable device, such as a stylus 103 arranged to 
co-operate with a touch tablet 104. In addition, the operator is also provided 
with a conventional keyboard 105. 

A typical image displayed on the VDU 102 is detailed in Figure 2. The 
display includes a canvas area 201 in which a visual image is created or an 
existing image is modified. Movement of stylus 103 is detected by system 
101 which in turn displays a cursor 202 arranged to map movements of the 
stylus 103 over the touch tablet 104. Cursor 202 may be moved into the 
canvas area 201 while performing "painting" operations. In addition, the 
cursor 202 may also be placed over soft buttons which are activated by 
placing the stylus 103 into pressure. In this way, particular types of operation 
may be effected by selecting soft buttons 203, 204, 205, 206 and 207. Thus 
conventional painting operations may be selected by these buttons, in which 
a new colour is applied to the image or. alternatively, an operation may be 
selected in which a control plane, or matte, is modified so as to change the 
way in which two image frames are blended. 

The image screen displayed on monitor 101 also includes a palette 
area 208 allowing particular colours to be selected, along with buttons 209, 
210, 211 and 212 allowing particular brush profiles to be selected. Thus, 
button 209 may select a sharp edged pencil, button 210 may select a softer 
edged paintbrush, button 211 may select a stippled or textured brush and 
button 210 may select a very smooth edged airbrush. 



Operational parameters of the system may be modrfied in response to 
activating system buttons, including buttons 213, 214 and 215. In this 
embodiment, the selection of button 215 allows the nature of the painting 
operation to be modrfied and results in the screen shown in Figure 2 being 
replaced by a screen of the type shown in Figure 3. 

The screen shown in Figure 3 again includes a plurality of buttons 
301, 302, 303, 304. 305, 306, 307, 308, 309, 310, 311, 312 and 313. During 
operation of the stylus 103 it is possible to vary the pressure applied to the 
stylus, modify the position of the stylus in the x direction and modify the 
position of the stylus in the y direction. These variables are expressed 
horizontally allowing relevant selections to be made from the matrix of 
buttons 301 to 313. These input variables may in turn adjust the intensity of 
the stamping, the size of the stamping, the rate at which stamping occurs and 
the texture of the stamping. 

In this way, when button 301 is activated, resulting in the way in which 
the button is represented being changed, the intensity of the brush stamp will 
vary in response to variations in applied pressure. These soft buttons are 
configured as toggles such that selecting button 301 again will result in the 
button being switched off such that intensity will not vary with pressure. 

Similarly, activation of button 302 will result in the intensity varying with 
respect to movement in the x direction and the selection of button 303 will 
result in variations occurring with respect to y directional movement. The 
selection of button 304 effectively cancels the operation of buttons 301 to 303 
and ensures that the intensity of modifications applied to the existing image 
remains constant, irrespective of input modifications. 

A similar situation is provided for adjusting the size of brush stamps. 
Thus, the activation of button 305 allows size to be modified with respect to 
pen pressure, the activation of button 306 allows size to be modified with 
respect to the x position, the activation of button 307 allows size to vary with 
respect to the y position and the activation of button 308 ensures that size 



remains constant irrespective of these variations. 

Thirdly, the rate at which stamps are applied to the image may be 
modified with respect to pressure by the activation of button 309, with 
variations in the x and y direction being controlled by the activation of buttons 
310 and 311. Again, the activation of button 312 ensures that the rate of 
stamp application remains constant irrespective of input variables. 

The application of texture to brush stamps introduces noise in 
accordance with the present invention. When button 313 is activated, noise 
will be introduced into the brush stamping from a tiled texture map, thereby 
mitigating the undesirable effects due to the scaling of brush profiles. Button 
313 is configured as a toggle and therefore once activated may be de- 
activated by a similar selection. 

Canvas 201 is illustrated in Figure 4, in which a first stroke 401 has 
been created by a plurality of textured stamps 402. The size of these stamps 
may be increased to produce a brush stamp of substantially similar profile but 
with a larger area of application. Thus, such a process may be implemented 
in order to create stroke 403 constructed from brush stamps 404 scaled up 
from the brush stamps used to create stroke 401 . 

Each brush stamp is derived as a texture which is rendered into a 
frame buffer at sub-pixel definition in response to the position of the stylus 
103 upon the touch tablet 104. Brush profiles are retained in memory in the 
form of a pixel patch which may consist of a square of 16 x 16 pixels, 32 x 32 
pixels, 64 x 64 pixels, or 128 x 128 pixels, etc. depending upon the definition 
of images being processed. A patch 501 of this type is illustrated in Figure 5, 
consisting of an array of 16 x 16 pixel values. In this embodiment, when 
larger brushes are required of substantially similar profile, pixel patch 501 is 
expanded to provide the number of pixels required. 

The patch 501 represents the two dimensional area of coverage in 
terms of pixel locations and the profile of the brush stamp is defined by the 
intensity values of the pixels, which may be represented as a surface 



extending vertically away from the plane. Such a surface is illustrated as a 
projection 502, which provides a basis for calculating actual pixel values 
contained within the patch. Thus, although the profile may be visualised as a 
smooth surface, given the digitised nature of the patch, the values actually 
contained within the patch have discrete values therefore the actual profile 
may be considered as that represented at 502. 

Profile 502 represents the profile of relevant pixels laying along line 
503. The pixel at location 504 is set to its maximum intensity identified as 
505. A similar intensity is allocated to pixels 506, 507 and 508. As pixel 
locations extend radially away from the centre, their intensities decrease. 
Thus, pixel locations 509, 510, 511, 512, 513, 514 and 515 have intensity 
values represented at 516, 517. 518, 519, 520, 521 and 522 respectively. 

Other profile shapes are stored representing different types of 
brushes, each with a pixel patch of 16 x 16 pixels. Generating a brush stamp 
which is four times larger in both the x and y dimensions, the 16 pixel x 16 
pixel patch is scaled up to a 64 pixel x 64 pixel patch. Each pixel patch 
contains fewer pixels than a full image frame and is arranged to be scaled to 
provide the size of patch required. Thus, a value from each pixel in the 
original 16x16 profile will be copied to 4 x 4 pixels in the scaled 64 x 64 pixel 
patch. 

In addition to storing a patch for each brush profile, a further patch is 
stored representing substantially random noise. A new texture stamp is 
created for being rendered within the image frame. The first stamping texture, 
representing the profile, is scaled which may result in the creation of 
undesirable artefacts. However, the second stamping texture, representing 
the substantially random noise, is expanded without scaling, preferably by a 
process of tiling. The scaled profile is then combined with the tiled noise to 
produce the new texture map in a way such that the tiled noise mitigates the 
effects of the undesirable artefacts introduced by the scaling of the profile. 
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Part of a scaled brush profile is illustrated in Figure 6. Each small 
square, such as square 601 , represents an individual pixel of the scaled 
texture. Each 4 pixel x 4 pixel square, identified by thickened boundaries 602, 
represents the regions scaled from a single pixel contained within the original 
profile. Thus, in this example, it may be assumed that region 602 is obtained 
from pixel position 514, region 603 is obtained from region 513 and region 
604 is obtained from pixel 512. Thus, all pixels 605 within regions 604 will 
have an intensity of 519. All pixels 606 within region 603 will have an intensity 
of 520 and all pixels 601 within regions 602 will have an intensity of 521. 

The tiling of noise is illustrated in Figure 7. In this example, it is 
assumed that the base tile consists of 4 pixels x 4 pixels although in practice 
the base tile would tend to contain significantly more pixels. Each pixel 701 
within a tile 702 has a substantially random value although the amplitude of 
these values will tend to be restrained within a relatively low range, such that 
the magnitude of the noise data is relatively small compared to the 
magnitude of the brush profile data. In this way, the noise may be considered 
as introducing a level of "dithering" to the brush profile and is not intended to 
substantially modify the brush profile. 

Pixel 701 has a particular noise value with pixel 702 having a 
randomly different value and again a different value is given to pixel 703 and 
so on. On the next tile, pixels at similar positions have similar values, 
introduced by effectively reproducing the base tile in a tiling operation. Thus, 
pixel 705, of tile 706 has the same value as pixel location 701 . 

Thus, it can be seen that the way in which brush profiles are 
expanded, as a scaling operation illustrated in Figure 6, differs from the way 
in which the noise is expanded, as a tiling operation, as illustrated in Figure 7. 
In this way, distortion artefacts introduced by each individual operation of 
tiling and the scaling of profiles do not mutually reinforce each other when the 
two expanded textures are combined, so that the combining process reduces 
unwanted artefacts arising from profile scaling. 



The overall procedure is illustrated in Figure 8. The process shows a 
situation in which a brush stamp is required which is larger than the stored 
profiles. At process 801 a brush profile, of the type shown in Figure 5, is 
scaled in the way described with reference to Figure 6. Process 802 
produces an expanded tiled random texture, invoking a process substantially 
similar to that described with respect to Figure 7. 

At process 803 the scaled brush texture from 801 and the tiled 
random texture from 802 are combined to produce a composite brusn stamp. 

The position of the brush stamp to sub-pixel definition is identified as 
804 and the brush stamp is rendered with an image at 805 under the control 
of a control value derived from 806, usually derived from stylus pressure. 

After rendering the new image at step 805 to determine new pixel 
values and filtering the image to take account of sub-pixel definition, the two 
images are combined as a modification process at step 806. 
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Claims 

1. Image data modifying apparatus, comprising image storage 
means, image display means, processing means and a manually controllable 
input means, wherein 

said storage means is arranged to store a plurality of modifying 
textures in the form of stamps having fewer pixels than a full image frame; 
and 

said processing means is configured to construct a new texture stamp 
by scaling a first stamping texture, expanding a second stamping texture 
without scaling and combining said scaled and said expanded textures. 

2. Apparatus according to claim 1, wherein said first stamping 
texture represents the brush profile. 

3. Apparatus according to claim 2, wherein said storage means is 
configured to store a plurality of brush profiles. 

4. Apparatus according to any of claims 1 to 3, wherein said 
second stamping texture represents random noise. 

5. Apparatus according to claim 4, wherein said random noise has 
a relatively low amplitude. 

6. Apparatus according to any of claims 1to 5, wherein a new 
texture is rendered into an image at sub-pixel definition. 

7. A method of modifying image data, in which pixel values in an 
image are modified in response to manual operation of an input device, 
comprising steps of storing a first stamping texture, storing a second 
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stamping texture and producing a new stamping texture by scaling said first 
stamping texture while expanding said second said second texture without 
scaling. 

8. A method according to claim 7 t wherein said first stamping 
texture represents a brush profile. 

9. A method according to claim 8, wherein a plurality of brush 
profiles are stored. 

10. A method according to claim 7, wherein said second stamping 
texture represents substantially random noise. 

11. A method according to claim 10, wherein said random noise 
has a relatively low amplitude. 

12. A method according to claim 7, wherein a new texture is 
rendered into an image at sub-pixel definition. 

13. A computer-readable medium having computer-readable 
instructions executable by a computer such that said computer performs the 
steps of: 

storing a first stamping texture in a first data field; 

storing a second stamping texture in a second data field; and 

modifying image data by producing an output stamping texture in 

response to scaling said first stamping texture while expanding said second 

stamping texture without scaling. 

14. A computer-readable medium having computer-executable 
instructions according to claim 13, wherein said first stamping texture 
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represents a brush profile. 

15. A computer-readable medium having computer-executable 
instructions according to claim 14, wherein a plurality of brush profiles are 

5 stored. 

16. A computer-readable medium having computer-executable 
instructions according to claim 13, wherein said second stamping texture 
represents substantially random noise. 

10 

17. A computer-readable medium having computer-executable 
instructions according to claim 16, wherein said random noise has a relatively 
low amplitude. 

15 18. A computer-readable medium having computer-executable 

instructions according to claim 13, wherein a new texture is rendered into an 
image at sub-pixel definition. 

19. A computer-readable storage system having a plurality of data 
20 fields stored on said medium and representing a data structure, comprising 

a first data field representing a first stamping texture; and 
a second data field representing a second stamping texture, such that, 
in response to image data modifying instructions an output stamping texture 
is produced by scaling said first texture and expanding said second texture 
25 without scaling. 

20. A storage system having a plurality of data fields according to 
claim 19, wherein said first stamping texture represents a brush profile. 
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21. Image data modifying apparatus, substantially as herein 
described with reference to the accompanying drawings. 



22. A method of modifying image data t substantially as herein 
5 described with reference to the accompanying drawings. 



